Method and apparatus for reducing inaccuracies when processing 
color data with a matrix 



5 RELATED APPLICATIONS 

This application is related to the application "Method and apparatus for 
reducing inaccuracies when processing color data with a tone map" that has the H.P. 
docket number 10016754 and was filed on the same day as this application. 

10 FIELD OF THE INVENTION 

The present invention relates generally to digital images and more 
specifically to a method and device for reducing inaccuracies when processing or 
creating color data. 

15 BACKGROUND OF THE INVENTION 

Digital cameras and scanners create image information by converting light 
into electrical signals using a photo-sensor. Typically the photo-sensors are charged 
coupled devices (CCD). The CCD's typically have 3 color filters arranged over the 
surface to sample the light in three different colors. Typically these are red, green, 

20 and blue. Scanners typically use an internal light source to illuminate the page. 
Cameras may use only the ambient illumination or they may add illumination to the 
scene using a flash. The electrical signal created by each pixel in the CCD is a 
function of the illumination in the scene, the reflectance of the object in the scene, 
the pass band of the filter over the pixel, and the inherent sensitivity of the CCD to 

25 light. One of the goals of these systems is to reproduce the color of an object as 
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perceived by the human eye. The human eye has a three-color detector system that 
senses red, green, and blue. Unfortunately it is difficult to match the three colors 
that the eye detects with the available sets of filters, CCD's, and illuminants. One 
method used to help match the machine detected colors with the human perceived 
5 colors is call matrix modification. The 3 colors from the CCD are used to create 3 
new output colors that more closely match the human perception. For example the 
resulting matrix modified red color may be generated by taking 85% of the red 
CCD output plus 25% of the green CCD output plus 7% of the blue CCD output 
(see figure 1). The ratios used for each matrix-modified color are typically 
10 dependent on the specific CCD, filter set and system performance for each type of 
camera or scanner. The matrix operation can be done in the scanner or camera 
hardware, or the raw color data may be sent to a computer where the image 
processing (including the matrix operation) is done. 

Some scanners and cameras may detect more than three different colors of 
15 light. For example some scanners may detect as many as 6 different colors. These 
additional colors can help the accuracy of the scanner or may be used to increase the 
number of colors detected. The output colors of these scanners may be a function of 
all 6 colors detected by the scanner. 

CCD's sample light and convert it into electrical signals. These electrical 
20 signals are then converted into digital information typically using an A-to-D 
converter. CCD's are typically fairly linear in response to the intensity of light 
falling on each pixel. The human eye is not linear in response to intensity. The 
human eye responds more to density, which is the log of intensity. Because of this 
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phenomenon a given change in intensity will be much more noticeable to the human 
eye if the change occurs in a dark area as compared to a light area of an image. For 
example in a system with 8 bits of resolution or 256 levels of intensity and 0 as 
black and 255 as white, a 2-count change in intensity from 10 to 12 may be much 
more noticeable to the human eye than the same 2-count change in intensity from 
200 to 202. 

The number of levels of light that the CCD and A-to-D converters can create 
has increased in recent years. Older scanners and cameras typically had 8 bits per 
color (256 levels) for a total of 24 bits. Today CCDs and A-to-D converters create 
10, 12 and even 16 bits per color. In addition to the increase in the number of 
levels, currently available CCDs have improved their signal-to-noise ratios. This 
means that more of the total number of levels are actual signal instead of noise. 
These changes have made reproductions of the dark areas of the images much more 
accurate. 

Unfortunately these changes have also created some problems. One of these 
problems is the matrix operation. In the dark areas of the image, inaccuracies in the 
matrix operation may show up as slight unintentional color changes in the image. 
The human eye may be unable to detect a slight change in brightness, but the human 
eye is very sensitive to small color shifts in the dark areas of an image. These small 
changes in color in the dark area of images can reduce the usability of these images. 

What is needed is a method and apparatus that reduces the inaccuracies in the 
matrix operation in the dark areas of the image. 
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SUMMARY OF THE INVENTION 

A method and device for improving the matrix operation in dark areas of an 

image. 

Other aspects and advantages of the present invention will become apparent 
5 from the following detailed description, taken in conjunction with the accompanying 
drawings, illustrating by way of example the principles of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram of a matrix operation for one output color. 
10 Figure 2 is a flow chart for the matrix operation in accordance with one 

embodiment of the current invention. 

Figure 3 is a flow chart for the matrix operation in accordance with another 
embodiment of the current invention. 

Figure 4 is a flow chart for the matrix operation in accordance with another 
15 embodiment of the current invention. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

A method and apparatus that reduces the inaccuracies in the matrix operation 
20 in the dark areas of the image can increase the number of usable images. 

The raw color components of a sample or pixel from the photo-sensor of 
scanners, cameras, or the like are typically mixed together using a matrix operation 
to produce a more accurate output color. The matrix operation creates new output 
color components for each pixel by mixing the raw color components from the 
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pixel. For example the output red component is composed from a ratio of the raw 
red, raw green, and raw blue color components. The output green component is 
composed from a different ratio of the raw red, raw green and raw blue color 
components. And the output blue color component would be composed of a third 
5 ratio of the three raw color components. Typically each new color component is 
composed mostly from the matching raw color component, i.e. the signal level of 
the output red color component is mainly determined by the raw red signal level. 

When the pixel or sample is from a dark area in the image, inaccuracies in 
the matrix operation may cause unacceptable color shifts for the pixel. The image 
10 can be improved by avoiding the color shifts in the dark areas of the image by 

selectively applying the matrix operation only to pixels that are not in a dark area of 
the image (see figure 2). The selective matrix operation can be performed in the 
hardware of the scanner, camera, color copier, or the like, or the raw color data can 
be transferred to a computer where the selective matrix operation can be performed. 
15 There are numerous methods to determine if a pixel is in a dark area of an 

image. In one embodiment a threshold is defined and if any raw color component in 
a pixel is greater than the threshold the pixel is considered to be in a lighter area of 
the image. In a preferred embodiment the threshold level or value is approximately 
10 eight-bit counts and in another preferred embodiment the threshold level or value 
20 is approximately 6 eight-bit counts. 

In another embodiment there may be a threshold for each raw color 
contained in a pixel. For example in a pixel that contains 3 color components (red, 
green, and blue) there may be a red threshold, a green threshold, and a blue 
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threshold. The pixel would be considered to be in a dark area only when the raw red 
color was below the red threshold and the raw green color was below the green 
threshold and the raw blue color was below the blue threshold. In another 
embodiment there may be three different raw color thresholds for each of the three 
5 output colors for a total of nine different thresholds. 

In another embodiment the threshold may be applied to the output color 
component instead of the raw color component. For example the threshold may be 
applied to the green output color created using the green ratio of raw color 
components, when the green output color is larger than the threshold the raw green 
10 color is used instead of the green output color. In another embodiment there may be 
a different threshold for each output color. 

A color described using red, green, and blue components mixes saturation, 
hue, and brightness together. Some color models have independent axis for 
brightness, for example CIE defines a color space where L* is the brightness axis. 
15 In these types of color spaces a single threshold used in the brightness axis may be 
used to determine when a pixel is in a dark area of the image. 

Once a pixel has been classified as being in a dark area of the image, the 
pixel can be treated in a number of ways. In one embodiment the pixels in the dark 
areas are left unchanged from the raw color photo-sensor output. In another 
20 embodiment the pixels in the dark areas of the image are transformed to a color 
using a second matrix. In another embodiment the pixels in the dark area of the 
image are converted to a shade of grey. Converting a color image to a shade of grey 
can be done using a second matrix, where each output color is created by the same 
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ratio of input color components. One matrix that can be used to convert color 
images into shades of grey is the NTSC matrix. 

A sharp transition between output pixels that are created using the matrix and 
output pixels that are in the dark areas may create image artifacts. It may be 
5 desirable to blend or smooth the transition between the dark area pixels and the 

pixels created using the main matrix (see figure 3). When a pixel is in a light area of 
the image (304) the normal matrix is used (306). If the pixel is in a transition area 
(308) the pixel may be blended or smoothed (310). When the pixel is in a dark area 
of the image the raw color component may be used (312). There are many ways to 
10 define which pixels are in a transition area. 

In one embodiment two thresholds are defined (see figure 4). In a preferred 
embodiment the upper threshold is 12 and the lower threshold is 8. Any pixel that 
has a color component above the higher threshold is not considered to be in the dark 
area. These pixels are transformed using the matrix (406). Pixels that have all their 
15 color components lower than the second threshold are left unmodified (412). Pixels 
that have all their color components below the higher threshold and have at least one 
color component between the two thresholds are considered to be in the transition 
area (410). In another embodiment pixels that have at least two color components 
between the two thresholds would be defined as being in the transition area. In 
20 another embodiment the raw color component that corresponds to the output color 
component would be required to be between the two thresholds before the pixel 
would be considered to be in the transition area. For example if the output color 
component was red, then the raw red color component would have to be between 
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the two thresholds for the pixel to be considered to be in the transition area. More 
than two thresholds can be used to create more than one transition area. All the 
pixels falling into each transition area could be treated in a different way. 

Once a pixel is defined as being in a transition area the pixel can be modified 

5 to help blend or smooth the transition between pixels in the dark areas and pixels not 
in the dark areas of the image. There are many ways to blend or smooth the 
transition area pixels. In one embodiment the pixels in the transition area are 
modified with a second matrix. In another embodiment the pixels in the transition 
area are modified by interpolation between the raw values and the transformed 

10 values from the matrix. For example if the high threshold is 12 out of 256 levels and 
the low threshold is 8, and a pixel has a raw red value of 1 1, a raw blue value of 6, 
and a raw green value of 4, and the red matrix values are .8 red, . 1 green, and .05 
blue, the matrixed red color component would be .8(11) + .1(4) + .05(6) = 9.5. 
The interpolated red value between the raw red value of 1 1 and the matrixed red 

15 value of 9.5 would be: (12-8)/(ll-8) = (9.5-ll)/(X-ll). Solving for X gives 9.875. 

The foregoing description of the present invention has been presented for 
purposes of illustration and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed, and other modifications and variations 
may be possible in light of the above teachings. The embodiment was chosen and 

20 described in order to clearly explain the principles of the invention and its practical 
application to thereby enable others skilled in the art to utilize the invention in 
various embodiments and various modifications as are suited to the particular use 
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contemplated. It is intended that the appended claims be construed to include other 
alternative embodiments of the invention except insofar as limited by the prior art. 
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